Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retry LB listener methods after timeout #8630

Merged
merged 3 commits into from
May 15, 2019
Merged

Conversation

ryndaniels
Copy link
Contributor

Community Note

  • Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
  • Please do not leave "+1" comments, they generate extra noise for pull request followers and do not help prioritize the request

Fixes #8025

Release note for CHANGELOG:

Adds a final retry to LB Listener requests if the requests originally timed out

Output from acceptance testing:

$ make testacc TESTARGS="-run=TestAccAWSLBListener"
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./... -v -parallel 20 -run=TestAccAWSLBListener -timeout 120m
?       github.com/terraform-providers/terraform-provider-aws   [no test files]
=== RUN   TestAccAWSLBListenerRule_basic
=== PAUSE TestAccAWSLBListenerRule_basic
=== RUN   TestAccAWSLBListenerRuleBackwardsCompatibility
=== PAUSE TestAccAWSLBListenerRuleBackwardsCompatibility
=== RUN   TestAccAWSLBListenerRule_redirect
=== PAUSE TestAccAWSLBListenerRule_redirect
=== RUN   TestAccAWSLBListenerRule_fixedResponse
=== PAUSE TestAccAWSLBListenerRule_fixedResponse
=== RUN   TestAccAWSLBListenerRule_updateRulePriority
=== PAUSE TestAccAWSLBListenerRule_updateRulePriority
=== RUN   TestAccAWSLBListenerRule_changeListenerRuleArnForcesNew
=== PAUSE TestAccAWSLBListenerRule_changeListenerRuleArnForcesNew
=== RUN   TestAccAWSLBListenerRule_multipleConditionThrowsError
=== PAUSE TestAccAWSLBListenerRule_multipleConditionThrowsError
=== RUN   TestAccAWSLBListenerRule_priority
=== PAUSE TestAccAWSLBListenerRule_priority
=== RUN   TestAccAWSLBListenerRule_cognito
=== PAUSE TestAccAWSLBListenerRule_cognito
=== RUN   TestAccAWSLBListenerRule_oidc
=== PAUSE TestAccAWSLBListenerRule_oidc
=== RUN   TestAccAWSLBListenerRule_Action_Order
=== PAUSE TestAccAWSLBListenerRule_Action_Order
=== RUN   TestAccAWSLBListenerRule_Action_Order_Recreates
=== PAUSE TestAccAWSLBListenerRule_Action_Order_Recreates
=== RUN   TestAccAWSLBListener_basic
=== PAUSE TestAccAWSLBListener_basic
=== RUN   TestAccAWSLBListener_BackwardsCompatibility
=== PAUSE TestAccAWSLBListener_BackwardsCompatibility
=== RUN   TestAccAWSLBListener_https
=== PAUSE TestAccAWSLBListener_https
=== RUN   TestAccAWSLBListener_Protocol_Tls
=== PAUSE TestAccAWSLBListener_Protocol_Tls
=== RUN   TestAccAWSLBListener_redirect
=== PAUSE TestAccAWSLBListener_redirect
=== RUN   TestAccAWSLBListener_fixedResponse
=== PAUSE TestAccAWSLBListener_fixedResponse
=== RUN   TestAccAWSLBListener_cognito
=== PAUSE TestAccAWSLBListener_cognito
=== RUN   TestAccAWSLBListener_oidc
=== PAUSE TestAccAWSLBListener_oidc
=== RUN   TestAccAWSLBListener_DefaultAction_Order
=== PAUSE TestAccAWSLBListener_DefaultAction_Order
=== RUN   TestAccAWSLBListener_DefaultAction_Order_Recreates
=== PAUSE TestAccAWSLBListener_DefaultAction_Order_Recreates
=== CONT  TestAccAWSLBListenerRule_basic
=== CONT  TestAccAWSLBListener_DefaultAction_Order_Recreates
=== CONT  TestAccAWSLBListener_basic
=== CONT  TestAccAWSLBListener_redirect
=== CONT  TestAccAWSLBListenerRule_Action_Order_Recreates
=== CONT  TestAccAWSLBListener_DefaultAction_Order
=== CONT  TestAccAWSLBListener_Protocol_Tls
=== CONT  TestAccAWSLBListener_https
=== CONT  TestAccAWSLBListener_BackwardsCompatibility
=== CONT  TestAccAWSLBListenerRule_Action_Order
=== CONT  TestAccAWSLBListenerRule_oidc
=== CONT  TestAccAWSLBListenerRule_cognito
=== CONT  TestAccAWSLBListenerRule_priority
=== CONT  TestAccAWSLBListenerRule_multipleConditionThrowsError
=== CONT  TestAccAWSLBListenerRule_changeListenerRuleArnForcesNew
=== CONT  TestAccAWSLBListenerRule_updateRulePriority
=== CONT  TestAccAWSLBListenerRule_fixedResponse
=== CONT  TestAccAWSLBListenerRule_redirect
=== CONT  TestAccAWSLBListenerRuleBackwardsCompatibility
=== CONT  TestAccAWSLBListener_cognito
--- PASS: TestAccAWSLBListenerRule_multipleConditionThrowsError (3.36s)
=== CONT  TestAccAWSLBListener_oidc
--- PASS: TestAccAWSLBListenerRuleBackwardsCompatibility (226.95s)
=== CONT  TestAccAWSLBListener_fixedResponse
--- PASS: TestAccAWSLBListenerRule_Action_Order_Recreates (228.69s)
--- PASS: TestAccAWSLBListener_cognito (232.67s)
--- PASS: TestAccAWSLBListener_BackwardsCompatibility (233.78s)
--- PASS: TestAccAWSLBListener_redirect (233.94s)
--- PASS: TestAccAWSLBListener_https (234.14s)
--- PASS: TestAccAWSLBListener_DefaultAction_Order (234.72s)
--- PASS: TestAccAWSLBListenerRule_oidc (235.91s)
--- PASS: TestAccAWSLBListenerRule_cognito (241.32s)
--- PASS: TestAccAWSLBListener_basic (243.72s)
--- PASS: TestAccAWSLBListener_DefaultAction_Order_Recreates (245.55s)
--- PASS: TestAccAWSLBListenerRule_Action_Order (249.00s)
--- PASS: TestAccAWSLBListener_oidc (249.84s)
--- PASS: TestAccAWSLBListenerRule_fixedResponse (257.92s)
--- PASS: TestAccAWSLBListenerRule_changeListenerRuleArnForcesNew (295.45s)
--- PASS: TestAccAWSLBListenerRule_updateRulePriority (306.47s)
--- PASS: TestAccAWSLBListenerRule_redirect (313.82s)
--- PASS: TestAccAWSLBListenerRule_basic (325.63s)
--- PASS: TestAccAWSLBListener_Protocol_Tls (353.97s)
--- PASS: TestAccAWSLBListener_fixedResponse (250.96s)
--- PASS: TestAccAWSLBListenerRule_priority (557.14s)
PASS
ok      github.com/terraform-providers/terraform-provider-aws/aws       558.022s

@ryndaniels ryndaniels requested a review from bflad May 14, 2019 15:40
@ghost ghost added size/XS Managed by automation to categorize the size of a PR. service/elbv2 Issues and PRs that pertain to the elbv2 service. labels May 14, 2019
@@ -493,6 +493,14 @@ func resourceAwsLbListenerCreate(d *schema.ResourceData, meta interface{}) error
return nil
})

if isResourceTimeoutError(err) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When the error is a resource.TimeoutError and we hit this condition, its important that we reset the error object so the later error checking logic is not triggered. Otherwise as written this will work like the following:

  1. resource.Retry sets top level err to resource.TimeoutError
  2. isResourceTimeoutError(err) conditional runs CreateListener again (great!)
  3. If CreateListener returns an error, it returns the error
  4. If CreateListener doesn't return an error, the isResourceTimeoutError() conditional exits, but the below error check (if err != nil) will still have the original resource.TimeoutError error, so it will still return an error

The trick here is using a direct = assignment in Go instead of :=

This allows us to simplify this logic like the following:

	// ... new logic here ...
	if isResourceTimeoutError(err) {
		_, err = elbconn.CreateListener(params)
	}

	// ... existing error handling below here ...
	if err != nil {
		return fmt.Errorf("Error creating LB Listener: %s", err)
	}

	// ... existing happy path ...

You'll now get the outcome you're expecting both ways when resource.Retry has a timeout error:

  1. resource.Retry sets top level err to resource.TimeoutError
  2. isResourceTimeoutError(err) conditional runs CreateListener again (great!)
  3. If CreateListener returns an error, it resets the top level error object and the below error check (if err != nil) will return that error
  4. If CreateListener doesn't return an error, the top level error object is reset to nil and the below error check (if err != nil) will not trigger so the resource will continue on its merry way on the happy path

Hope this helps!

@bflad
Copy link
Contributor

bflad commented May 14, 2019

edit: Whoops I thought the original issue reference was the tech debt issue (#7873), but it might be good to reference that one as well. 😅

Copy link
Contributor

@bflad bflad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@ryndaniels ryndaniels merged commit 5e7c8ff into master May 15, 2019
@ryndaniels ryndaniels deleted the rfd-retry-timeout branch June 20, 2019 11:44
@ghost
Copy link

ghost commented Nov 3, 2019

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Nov 3, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
service/elbv2 Issues and PRs that pertain to the elbv2 service. size/XS Managed by automation to categorize the size of a PR.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

aws_lb_listener read taking longer than 1 minute timeout
2 participants